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Background: Evaluating diversity, inclusivity, and equity 
remains both a prevalent topic in education and a difficult 
challenge for most evaluators. Traditional metrics used to 
evaluate these constructs include questionnaires, focus 
groups, and anonymous comment solicitations. While each of 
these approaches offer value, they also possess a number of 
limitations (e.g., self-reported nature, holistic perspective, 
social desirability bias, varying degrees of respondent 
sensitivity, representative responses, etc.). Researchers at the 
University of Southern California have successfully utilized the 
Equity Index Method (EIM) as a potential approach for 
measuring diversity and reporting diversity-related outcomes. 

Purpose: Provide a critique of the EIM and discusses how the 
EIM could be improved and extended to other evaluation 
contexts and settings. 


Setting: Not Applicable. 

Intervention: Not Applicable. 

Research Design: Not Applicable. 

Data Collection and Analysis: Not Applicable. 

Findings: Despite the potential for problems with 

interpretations based on small samples and subgroups and 
some concerns about semantics involving the term "equity", 
we believe the EIM possesses a number of strengths that 
many evaluators will find useful. We encourage other 
evaluators to consider this method and explore its utility in a 
variety of contexts. 
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Introduction 

Evaluating diversity, inclusivity, and equity 
remains both a prevalent topic in education and a 
difficult challenge for most evaluators. Traditional 
metrics used to evaluate these constructs include 
questionnaires (e.g., campus climate surveys, 
senior exit surveys, etc.), focus groups, and 
anonymous comment solicitations. While each of 
these approaches offer value, they also possess a 
number of limitations (e.g., self-reported nature, 
holistic perspective, social desirability bias, 
varying degrees of respondent sensitivity, 
representative responses, etc.) As persons 
interested in evaluating this challenging construct 
we were intrigued by the Equity Index Method 
(EIM) (Hao, 2002; Bensimon, Bustillos & Hao, 
2006; Hao and Malcom-Piqueux, 2014) as it 
provides a simple calculation that could be used to 


discern matters of diversity, inclusivity and equity 
in a variety of settings. Hao (2002) and Bensimon, 
Bustillos & Hao (2006) define the EIM as “a 
measure of proportionality that assesses the extent 
to which some population of interest is equitably 
represented among individuals who have achieved 
a specific educational outcome”. Researchers at 
the University of Southern California (USC) have 
successfully utilized the EIM to analyze education 
data on a variety of indicators, such as access (e.g., 
enrollment), outcomes (e.g., degrees awarded), 
and excellence (e.g., high-value and high-priority 
fields and programs). 

Equity Index Method 

The Equity Index Method can be calculated via the 
following formula: 


Target Group's Equity Index for 
the educational outcome of 
interest 


Target group with the educational outcome -r 
Total students with the educational outcome 
Target group in the reference population -r 
Total students in the reference population 


Because equity is based on population 
proportions, Hao (2002) proposed the following 
interpretation guide: 


Table 1 


Equity Index Interpretation Guide 


Performance Level 

Equity Index Value 

Description 

High Performance 

Greater than or equal to 1 

At or above equity 

Medium-High Performance 

0.85 < Equity Index <0.99 

Almost at equity 

Medium-Low Performance 

0.70 < Equity Index < 0.85 

Below equity 

Low Performance 

Equity Index < 0.70 

Far below equity 


According to the proposed interpretation guide, 
values that approximate 1.0 represent equity, a 
score less than 1.0 is below equity, and a score 
greater than 1.0 is above equity. 

Bensimon et al. (2006) offered the 
following fictitious example to illustrate the EIM 
and equity index interpretation. Suppose a 
graduating class of a high school consists of 1,000 


students, with 400 (40%) being Latino. Now, 
suppose a total of 450 of these students enroll in 
the state’s flagship university, but only 45 (10%) 
are Latino. When the data are substituted into the 
formula, a measure of .25 is obtained. This value 
indicates the proportion of Latino students 
attending the flagship university is far below 
equity. 


Latino students' Equity Index 
for attending the flagship 
university 


45 Latino students enrolled in college t 
450 total cohort college enrollment 
400 Latino high school graduates t 
1,000 total high school graduates 


10% 

= 0.25 

40% 
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For convenience, researchers at USC have 
published an online calculator at 

http://cue.usc.edu/hsi/eq.html for others to 
substitute values and produce equity calculations. 
Also on the website is a PowerPoint presentation 
with several useful examples of the EIM in 
practice. 

Strengths of the Equity Index Method 

The EIM possesses a number of strengths. As 
noted previously, the ability to apply the formula 
to a variety of indicators (e.g., enrollment, 
retention, degrees awarded, etc.) is particularly 
powerful. The EIM may also serve as a useful tool 
for internal benchmarking purposes, as 
longitudinal comparisons can be made to assess 
whether various demographic groups are 
performing in equitable ways. Further, the method 
is also appealing for its ease of use. For example, 
anyone with programmatic data (including 
demographic variables) can easily use the 
calculator to quantify potential equity gaps. The 
EIM may be a preferred alternative to statistical 


approaches (e.g., chi-squared test, 2-proportion z- 
test, etc.) that attempt to investigate whether 
proportional differences exist between two groups. 

Weaknesses of the Equity Index 
Method 

Despite the aforementioned advantages, the EIM 
possesses a major vulnerability with regard to 
small samples and subgroups. This is particularly 
problematic given a significant amount of work 
with quantitative data involves small samples. 
Below, we will demonstrate the vulnerability of 
this method: 

Suppose a classroom consists of 25 students: 
20 (80%) are White and 5 (20%) are students of 
color. Suppose 17 of the 20 (85%) White students 
complete the course, and 3 of the 5 students of 
color (60%) complete the course. When we enter 
the appropriate values into the calculator we are 
given an Equity Index value of .75, indicating the 
students of color performed “below equity”. 


3 students of color complete course t 
S tudents of color Equity _ 20 total students complete the course 

Index for course completion 5 students of color enrolled in course ■* 

25 total students enrolled in course 

Now, suppose 4 of 5 (80%) students of color into the calculator we are given an Equity Index 

successfully completed the course, and 16 of 20 value of 1.00, indicating the students of color 

(80%) White students also successfully completed performed “at or above equity”, 
the course. When we enter the appropriate values 


15% 

= 0.75 

20% 


4 students of color complete course -r 
Students of color Equity _ 20 total students complete the course 

Index for course completion 5 students of color enrolled in course -r 

25 total students enrolled in course 


20% 

= 1.00 

20% 


This example underscores the problem of 
proportional sensitivity with small samples. The 
performance of a single student from the target 
group caused the interpretation to jump from 
“below equity” to “at or above equity”, a stark 
contrast in results. Because of this vulnerability, 
we recommend the authors provide a note 
cautioning users about the inferential validity of 
index measures obtained from small group sizes, 
as the consequences of making inappropriate 
inferences about a program or subpopulation 
group can be devastating (e.g., programs may 
undergo an unnecessary redesign, receive funding 


cuts, etc.). One suggestion to help ensure more 
valid inferences are made would be to offer 
guidelines based on either the smallest 
recommended sample size, or perhaps a range of 
sample sizes, as this would help ensure evaluators 
are not accidentally lead to potentially misleading 
conclusions. 

A second significant concern with this method 
involves semantics, as the term “equity” may not 
be appropriate. In the context of social and 
diversity measurement the definition of equity 
typically pertains to fairness issues. Thus, simply 
because outcomes may result in proportional 
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differences across subpopulations does not 
necessarily imply issues of fairness had any 
influence on the results. We believe the term 
“equity” possesses additional connotations that 
could have unintended consequences for the use of 
this method. One prime example is the decreased 
likelihood that others will extend this method to 
other areas of assessment and evaluation simply 
because of its naming convention. We believe a 
more neutrally-charged and descriptive term for 
this method may be the “Proportional Differences 
Index”, or something similar. A more generic 
name to describe what is really taking place with 
the calculation would likely increase the visibility 
of this method, and its application to a variety of 
evaluation contexts. 

Potential Applications of the Equity 
Index Method in Evaluation 

As noted previously, the USC team of researchers 
have published numerous examples using the EIM 
with mostly Hispanic-serving institutions. 
However, the methodology could be easily 
extended to a number of additional scenarios. For 
example, the outcomes of virtually any subgroup 
could be evaluated based on a particular 
demographic characteristic (e.g., race/ethnicity, 
sex, gender, in-State vs. out-of-state residency 
status, household income levels, parents’ highest 
education level, first generation college student 
status, etc.). This makes the method particularly 
appealing as it may have broad applicability for 
evaluators. 

An “Equity Index” value (or perhaps a 
“Proportional Demographic Outcomes Index”), 
possibly as a color-coded marker, would likely 
make a nice complement to traditional descriptive 
statistics on a score report, such as a dashboard or 
report card. The ability to quickly calculate an 
index of proportional outcomes success may be 
more useful than simply visualizing descriptive 
statistics alone. In the context of higher education 
diversity outcome metrics, this method could be 
useful for discerning proportional successes by 
various subpopulation groups. In the case of 
examination score reporting, this method could 
provide some quick diagnostics to assist evaluators 
with tests for item bias (e.g., Differential Item 
Functioning (DIF)). In the context of program 
evaluation, the method could be used to determine 
markers of success, such as performance levels, by 
a wide array of participant characteristics. 


Conclusion 

Despite the potential for problems with 
interpretations based on small samples and 
subgroups and some concerns about semantics 
involving the term “equity”, we believe this 
methodology possesses a number of strengths that 
many evaluators will find useful. We encourage 
other evaluators to consider this method and 
explore its utility in a variety of contexts. 
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